The GitHub Action workflow was failing due to timeouts and job cancellations. This PR implements fixes to ensure reliable workflow execution.

Root Cause

Issue 1: "Fix modification dates" step causing 4+ hour network timeouts

The snakemake --touch resources/regions/${{ matrix.sample }}/test-regions.cov-*.bed command builds a full dependency DAG, including upstream rules that download large sample files:

- name: Fix modification dates
  uses: snakemake/snakemake-github-action@v2.0.3
  with:
    args: "--cores 1 --sdm conda --touch resources/regions/${{ matrix.sample }}/test-regions.cov-*.bed"

This triggered network failures: ChunkedEncodingError: Connection broken: IncompleteRead(4313039036 bytes read, 1279940331 more expected)

Issue 2: Jobs hitting GitHub's 6-hour time limit

After initial chunking with 8 variant calls per job, some jobs were still taking 3-6 hours and hitting GitHub Actions' hard 6-hour job time limit. Analysis of workflow run 21831716746 showed that jobs processing 8 variant calls plus alignment exceeded time limits.

Changes

1. Removed "Fix modification dates" step entirely

   • The --rerun-triggers mtime flag in the "Run analysis" step already handles timestamp-based reruns
   • Git-tracked region files don't require timestamp adjustment

2. Implemented chunk-based workflow splitting with optimized chunk size

   • Modified setup job to dynamically analyze variant call counts per benchmark
   • Initially split large benchmarks into chunks of 8 variant calls each
   • After analyzing failures, reduced chunk size to 4 variant calls for better distribution
   • Added chunk filtering logic in workflow/rules/common.smk using chunk_index and chunk_size parameters
   • Updated artifact names to include chunk identifiers

3. Made report job tolerant of partial failures

   • Added if: ${{ !cancelled() && (success() || failure()) }} condition to report job
   • Report will now generate even if some evaluate jobs fail
   • Uses specific artifact pattern (giab-*) to download only benchmark results

Results

Before: 3 matrix jobs

• giab-NA12878-agilent-200M: 23 variant calls (single job)
• giab-NA12878-agilent-75M: 14 variant calls (single job)
• giab-NA12878-twist: 6 variant calls (single job)

After: 12 matrix jobs (with chunk_size=4)

• giab-NA12878-agilent-200M: Split into 6 jobs (4, 4, 4, 4, 4, 3 variant calls)
• giab-NA12878-agilent-75M: Split into 4 jobs (4, 4, 4, 2 variant calls)
• giab-NA12878-twist: Split into 2 jobs (4, 2 variant calls)

Each job now processes at most 4 variant calls, ensuring all jobs complete within GitHub's 6-hour limit and preventing timeout/cancellation issues. The report job generates successfully even if some individual benchmarks fail.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.